Skip to content

Emit LIMIT -1 when offset is used without limit - #821

Merged
simonw merged 2 commits into
simonw:mainfrom
ethanhawkes-gif:fix-offset-without-limit
Aug 12, 2026
Merged

simonw merged 2 commits into
simonw:mainfrom
ethanhawkes-gif:fix-offset-without-limit

Conversation

@ethanhawkes-gif

@ethanhawkes-gif ethanhawkes-gif commented Aug 1, 2026

Copy link
Copy Markdown
Contributor

Fixes:

SQLite requires a LIMIT clause to appear before OFFSET, so passing offset without limit generated invalid SQL:

db["t"].rows_where(offset=2)
# select * from "t" offset 2
# OperationalError: near "2": syntax error

A negative limit means "no upper bound" in SQLite, so emitting limit -1 offset N returns all rows from position N onwards.

The issue mentions rows_where() and search_sql(). There is a third instance of the same pattern in the sqlite-utils rows CLI command, which fails the same way:

$ sqlite-utils rows data.db t --offset 1
Error: near "1": syntax error

So this fixes three sites:

  • Queryable.rows_where() — also covers pks_and_rows_where(), which delegates to it
  • Table.search_sql() — also covers search()
  • the sqlite-utils rows CLI command

Tests added:

  • two new cases in the existing test_rows_where_offset_limit parametrize (offset with limit=None, including offset=0)
  • test_pks_and_rows_where_offset_without_limit
  • test_search_offset_without_limit
  • a new --offset without --limit case in the sqlite-utils rows CLI parametrize

Full suite passes (1379 passed, 16 skipped), plus black --check, flake8 and mypy are clean.


📚 Documentation preview 📚: https://sqlite-utils--821.org.readthedocs.build/en/821/

SQLite requires a LIMIT clause to appear before OFFSET, so passing offset
without limit generated invalid SQL such as:

    select * from "t" offset 2

which raised OperationalError: near "2": syntax error.

A negative limit means "no upper bound" in SQLite, so "limit -1 offset N"
returns all rows from position N onwards.

Fixed in three places that build LIMIT/OFFSET SQL:

- Queryable.rows_where() - also covers pks_and_rows_where()
- Table.search_sql() - also covers search()
- the "sqlite-utils rows" CLI command
@ethanhawkes-gif

Copy link
Copy Markdown
Contributor Author

Friendly ping on this one — it's mergeable and CI is green, and I'd rather have a clear no than
leave it sitting.

The change is narrow: when offset is passed without limit, emit LIMIT -1 so SQLite accepts
the query instead of raising a syntax error. Per the SQLite SELECT docs
(https://sqlite.org/lang_select.html): "If the LIMIT expression evaluates to a negative value,
then there is no upper bound on the number of rows returned." Generated SQL is unchanged whenever
a limit is already present.

Any of these work for me — merge, request changes, or close it as intended behaviour. Just say
which and I'll handle it.

@simonw

simonw commented Aug 12, 2026

Copy link
Copy Markdown
Owner

Thanks for this!

@simonw
simonw merged commit 43d5d33 into simonw:main Aug 12, 2026
1 check was pending
simonw added a commit that referenced this pull request Aug 12, 2026
simonw added a commit that referenced this pull request Aug 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants